Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems

International audienceThe finite-size ensemble Kalman filter (EnKF-N) is an ensemble Kalman filter (EnKF) which, in perfect model condition, does not require inflation because it partially accounts for the ensemble sampling errors. For the Lorenz '63 and '95 toy-models, it was so far shown to perform as well or better than the EnKF with an optimally tuned inflation. The iterative ensemble Kalman filter (IEnKF) is an EnKF which was shown to perform much better than the EnKF in strongly nonlinear conditions, such as with the Lorenz '63 and '95 models, at the cost of iteratively updating the trajectories of the ensemble members. This article aims at further exploring the two filters and at combining both into an EnKF that does not require inflation in perfect model condition, and which is as efficient as the IEnKF in very nonlinear conditions. In this study, EnKF-N is first introduced and a new implementation is developed. It decomposes EnKF-N into a cheap two-step algorithm that amounts to computing an optimal inflation factor. This offers a justification of the use of the inflation technique in the traditional EnKF and why it can often be efficient. Secondly, the IEnKF is introduced following a new implementation based on the Levenberg-Marquardt optimisation algorithm. Then, the two approaches are combined to obtain the finite-size iterative ensemble Kalman filter (IEnKF-N). Several numerical experiments are performed on IEnKF-N with the Lorenz '95 model. These experiments demonstrate its numerical efficiency as well as its performance that offer, at least, the best of both filters. We have also selected a demanding case based on the Lorenz '63 model that points to ways to improve the finite-size ensemble Kalman filters. Eventually, IEnKF-N could be seen as the first brick of an efficient ensemble Kalman smoother for strongly nonlinear systems

Joint state and parameter estimation with an iterative ensemble Kalman smoother

International audienceBoth ensemble filtering and variational data assimilation methods have proven useful in the joint estimation of state variables and parameters of geophysical models. Yet, their respective benefits and drawbacks in this task are distinct. An ensemble variational method, known as the iterative ensemble Kalman smoother (IEnKS) has recently been introduced. It is based on an adjoint model-free variational, but flow-dependent, scheme. As such, the IEnKS is a candidate tool for joint state and parameter estimation that may inherit the benefits from both the ensemble filtering and variational approaches. In this study, an augmented state IEnKS is tested on its estimation of the forcing parameter of the Lorenz-95 model. Since joint state and parameter estimation is especially useful in applications where the forcings are uncertain but nevertheless determining, typically in atmospheric chemistry, the augmented state IEnKS is tested on a new low-order model that takes its meteorological part from the Lorenz-95 model, and its chemical part from the advection diffusion of a tracer. In these experiments, the IEnKS is compared to the ensemble Kalman filter, the ensemble Kalman smoother, and a 4D-Var, which are considered the methods of choice to solve these joint estimation problems. In this low-order model context, the IEnKS is shown to significantly outperform the other methods regardless of the length of the data assimilation win- dow, and for present time analysis as well as retrospective analysis. Besides which, the performance of the IEnKS is even more striking on parameter estimation; getting close to the same performance with 4D-Var is likely to require both a long data assimilation window and a complex modeling of the background statistics

Application of a hybrid EnKF-OI to ocean forecasting

Data assimilation methods often use an ensemble to represent the background error covariance. Two approaches are commonly used; a simple one with a static ensemble, or a more advanced one with a dynamic ensemble. The latter is often non-practical due to its high computational requirements. Some recent studies suggested using a hy- brid covariance, which is a linear combination of the covariances represented by a static and a dynamic ensemble. Here, the use of the hybrid covariance is first extensively tested with a quasi-geostrophic model and with different analysis schemes, namely the Ensemble Kalman Filter (EnKF) and the Ensemble Square Root Filter (ESRF). The hybrid covariance ESRF (ESRF-OI) is more accurate and more stable than the hybrid covariance EnKF (EnKF-OI), but the overall conclusions are similar regardless of the analysis scheme used. The benefits of using the hybrid covariance are large compared to both the static and the dynamic methods with a small dynamic ensemble. The benefits over the dynamic methods become negligible, but remain, for large dynamic ensembles. The optimal value of the hybrid blending coefficient appears to decrease exponentially with the size of the dynamic ensemble. Finally, we consider a realistic application with the assimilation of altimetry data in a hybrid coordinate ocean model (HYCOM) for the Gulf of Mexico, during the shedding of Eddy Yankee (2006). A 10-member EnKF-OI is compared to a 10-member EnKF and a static method called the Ensemble Optimal Interpolation (EnOI). While 10 mem- bers seem insufficient for running the EnKF, the 10-member EnKF-OI reduces the forecast error compared to the EnOI, and improves the positions of the fronts

Asynchronous data assimilation with the EnKF in presence of additive model error

The term ‘asynchronous data assimilation’ (ADA) refers to modifications of sequential data assimilation methods that take into consideration the observation time. In Sakov et al. [Tellus A, 62, 24–29 (2010)], a simple rule has been formulated for the ADA with the ensemble Kalman filter (EnKF). To assimilate scattered in time observations, one needs to calculate ensemble forecast observations using the forecast ensemble at observation time. Using then these ensemble observations in the EnKF update matches the optimal analysis in the linear perfect model case. In this note, we generalise this rule for the case of additive model error

An iterative ensemble Kalman smoother

International audienceThe iterative ensemble Kalman filter (IEnKF) was recently proposed in order to improve the performance of ensemble Kalman filtering with strongly nonlinear geophysical models. The IEnKF can be used as a lag-one smoother and extended to a fixed-lag smoother: the iterative ensemble Kalman smoother (IEnKS). The IEnKS is an ensemble variational method. It does not require the use of the tangent linear of the evolution and observation models, nor the adjoint of these models: the required sensitivities (gradient and Hessian) are obtained from the ensemble. Looking for optimal performance, out of the many possible extensions we consider a quasi-static algorithm. The IEnKS is explored for the Lorenz '95 model and for a two-dimensional turbulence model. As the logical extension of the IEnKF, the IEnKS significantly outperforms standard Kalman filters and smoothers in strongly nonlinear regimes. In mildly nonlinear regimes (typically synoptic-scale meteorology), its filtering performance is marginally but clearly better than the standard ensemble Kalman filter and it keeps improving as the length of the temporal data assimilation window is increased. For long windows, its smoothing performance outranks the standard smoothers very significantly, a result that is believed to stem from the variational but flow-dependent nature of the algorithm. For very long windows, the use of a multiple data assimilation variant of the scheme, where observations are assimilated several times, is advocated. This paves the way for finer reanalysis, freed from the static prior assumption of 4D-Var but also partially freed from the Gaussian assumptions that usually impede standard ensemble Kalman filtering and smoothing

An adaptive quality control procedure for data assimilation

We describe a simple adaptive quality control procedure that limits the impact of individual observations likely to be inconsistent with the state of the data assimilation system. It smoothly increases the observation error variance depending on the projected increment, state error variance and so-called K-factor so that the resulting increment does not exceed the estimated state error times K. Because an estimate of the state error is readily available in the Kalman filter (KF), the method is particularly suitable for the KF, ensemble Kalman filter (EnKF), or ensemble optimal interpolation systems. The tests show that setting K to about 1.5–2 or above has no detrimental effect for performance of nearly optimal systems; at the same time it still makes it possible to make use of observations that might otherwise be discarded by the background check. The technique is successfully used in the EnKF codes TOPAZ and EnKF-C